A computational model of discourse predictions in sentence processing

Recent research in psycholinguistics has seen a growing interest in the role of prediction in sentence processing. Most attempts to computationally model predictive processing have focused on syntactic prediction. Examples include Hale (2001)’s surprisal model, which relates processing effort to the conditional probability of the current word given the previous words in the sentence. Recent work has attempted to integrate semantic and discourse prediction with models of syntactic processing. This includes Mitchell et al. (2010)’s approach, which combines an incremental parser with a vector-space model of semantics. However, this approach only provides a loose integration of the two components, and the notion of semantics is restricted to lexical meaning approximated by word co-occurrences. At the discourse level, Dubey (2010) has proposed a model that combines an incremental parser with a probabilistic logic-based model of co-reference resolution. However, this model does not explicitly model discourse effects in terms of prediction, and again only proposes a loose integration of co-reference and syntax. Furthermore, the Dubey (2010) model has not been evaluated on broad coverage data.